Working range and lift detection in an input device

ABSTRACT

A method of operating an input device can include generating a light beam by a light source module, steering the light beam towards a target location on an underlying surface, steering a reflected light beam towards an image sensor of the input device, receiving the reflected light beam by the image sensor, and generating tracking data by the image sensor that corresponds to a two-dimensional (2D) movement of the input device on the underlying surface. The method further includes determining that the input device is operating: on and in contact with the underlying surface when the reflected light beam received by the image sensor is located on a first set of pixels of the image sensor, and above and not in contact with the underlying surface when the reflected light beam is located on a second set of pixels of the plurality of pixels of the image sensor.

BACKGROUND

Input devices are commonplace in modern society and are typically usedto convert human-induced analog inputs (e.g., touches, clicks, motions,touch gestures, button presses, scroll wheel rotations, etc.) made inconjunction with an input device into digital signals for computerprocessing. An input device can include any device that can provide dataand control signals to a computing system. Some non-limiting examples ofinput devices include computer mice, keyboards, virtual reality and/oraugmented reality controllers, touch pads, remote controls, gamingcontrollers, joysticks, trackballs, and the like. Some non-limitingexamples of computing systems include desktops, laptops, tablets and“phablet” computers, smart phones, personal digital assistants, wearabledevices (e.g., smart watches, glasses), virtual reality (VR) and/oraugmented reality (AR) systems, and the like.

Computer mice, in particular, have undergone significant improvements infunctionality, accuracy, ergonomics, and versatility. Earlier designs,including the “mechanical mouse,” used a rubber ball coupled to twofreely rotating rollers situated 90 degrees from one another to rollalong an underlying surface. The first roller detects forward-backwardmotion of the mouse and the second roller detects left-right motion,with each roller sharing the same shaft as a corresponding encoder wheelwith slotted edges that interrupt infra-red light beams generateelectrical pulses that can be translated to wheel movement, Mechanicalmice were notorious for picking up dirt, unpredictable tracking, andneeding frequent disassembly and cleaning.

Contemporary mice may include optical mice using optoelectronic sensorsto compare successive images of an underlying surface on which thecomputer mouse operates to interpret movement. Technologicalimprovements have allowed optical mice to functionally track over variedtypes of surfaces (e.g., table tops, paper, glass, etc.), while avoidingsome of the problems associated with mechanical mice. Optical micetypically employ light-emitting diodes (LEDs) and/or laser (e.g.coherent) light and an imaging array of photodiodes to detect movementrelative to the underlying surface, which has proven to be much morereliant and robust as compared to their mechanical counterparts.Multi-surface use allows usage over a wider range of applications, whichcan be desirable by the average consumer. Despite these advantages, moreimprovements are needed for the more discerning consumers.

It should be noted that unless otherwise indicated herein, the materialsdescribed in this section are not prior art to the claims in thisapplication and are not admitted to be prior art by inclusion in thissection.

BRIEF SUMMARY

In certain embodiments, an input device comprises a housing; one or moreprocessors; a light source module coupled to the housing and controlledby the one or more processors, the light source module configured togenerate and direct light towards an underlying surface that the inputdevice is operating on; and an image sensor module coupled to thehousing and controlled by the one or more processors. The image sensormodule can include an image sensor configured to receive reflected lightfrom the light source module that is reflected off of the underlyingsurface, and generate tracking data that corresponds to atwo-dimensional (2D) movement of the input device with respect to anunderlying surface based on the received reflected light from the lightsource module. The image sensor can be comprised of a plurality ofpixels including a first set of pixels of the plurality of pixelsconfigured to receive the reflected light from the light source modulewhen the input device is operating on the underlying surface, and asecond set of pixels of the plurality of pixels adjacent to the firstset of pixels that is configured to extend a vertical movement detectionrange of the input device by receiving the reflected light from thelight source module when the input device is lifted off of theunderlying surface. In some implementations, the first set of pixelsforms a square shape that receives the reflected light from the lightsource module when the input device is operating on the underlyingsurface, where the second set of pixels is adjacent to the first set ofpixels such that the first set of pixels and the second set of pixelstogether form a rectangle, and wherein the second set of pixels isconfigured at a location relative to the first set of pixels such thatthe reflected light from the light source module moves from the firstset of pixels to the second set of pixels as the input device is liftedoff of the underlying surface. In some cases, the input device isconfigured to detect both 2D movement of the input device relative tothe underlying surface and detect the input device being lifted off ofthe corresponding surface using a single image sensor module (e.g.,system 500).

In further embodiments, reflected light from the light source moduleforms a spot on the first set of pixels when the input device isoperating on the underlying surface, and wherein the one or moreprocessors are configured to detect an edge of the spot by identifyingboundaries where a first pixel of a pair of adjacent pixels are at orabove a threshold illumination value and a second pixel of a pair ofadjacent pixels are below the threshold illumination value, anddetermine a centroid of the spot based on the detected edge of the spot.The one or more processors can be further configured to determine anamount that the input device has lifted off of the underlying surfacebased on the location of the determined centroid of the spot on theplurality of pixels. Some embodiments may include an inertialmeasurement unit (IMU) with an accelerometer, where the one or moreprocessors are further configured to determine whether the input devicehas been lifted vertically from the underlying surface or tilted off ofthe underlying surface based, in part, on inertial data received fromthe IMU and the location of the centroid of the spot on the plurality ofpixels. The light source module can include an infra-red LED. In somecases, the input device can further include a first lens configured todirect light from the light source module towards the underlying surfaceand a second lens configured to direct the reflected light off of theunderlying surface to the first set of pixels of the image sensor whenthe input device is operating on the underlying surface. In some cases,the reflected light substantially overfills the first set of pixels andnot the second set of pixels while the input device is operating on theunderlying surface, the reflected light substantially fills at least amajority portion of the second set of pixels when the input device islifted or tilted off of the underlying surface.

Some embodiments can include a method of operating an input device, themethod comprising: generating a light beam by a light source modulecontrolled by one or more processors of the input device; steering thelight beam towards a target location, wherein the target locationcorresponds to a spot on an underlying surface while the input device isoperating on the underlying surface; steering a reflected light beamthat is reflected off of the underlying surface towards an image sensorof the input device; receiving the reflected light beam by the imagesensor, the image sensor controlled by the one or more processors;generate tracking data by the image sensor that corresponds to atwo-dimensional (2D) movement of the input device with respect to theunderlying surface based on the received reflected light beam; determinethat the input device is operating on and in contact with the underlyingsurface when the reflected light beam received by the image sensor islocated on a first set of pixels of a plurality of pixels of the imagesensor; and determine that the input device is operating above and notin contact with the underlying surface when the reflected light beamreceived by the image sensor is located on a second set of pixels of theplurality of pixels of the image sensor. In some implementations, thefirst set of pixels forms a square shape that receives the reflectedlight from the light source module when the input device is operating onthe underlying surface, where the second set of pixels is adjacent tothe first set of pixels such that the first set of pixels and the secondset of pixels together form a rectangle, and where the second set ofpixels is configured at a location relative to the first set of pixelssuch that the reflected light from the light source module moves fromthe first set of pixels to the second set of pixels as the input deviceis lifted off of the underlying surface. The light source module canform a spot on the first set of pixels when the input device isoperating on the underlying surface, and the method can furthercomprise: detecting an edge of the spot, by the one or more processors,by identifying boundaries where a first pixel of a pair of adjacentpixels are at or above a threshold illumination value and a second pixelof a pair of adjacent pixels are below the threshold illumination value;and determining a centroid of the spot, by the one or more processors,based on the detected edge of the spot.

In further embodiments, the method can further include determining, bythe one or more processors, an amount that the input device has liftedoff of the underlying surface based on the location of the determinedcentroid of the spot on the plurality of pixels. The input device mayfurther comprises an IMU with an accelerometer, wherein the methodfurther includes: determining whether the input device has been liftedvertically from the underlying surface or tilted off of the underlyingsurface based, in part, on inertial data received from the IMU and thelocation of the centroid of the spot on the plurality of pixels. In someembodiments, the input device further includes an illumination lens andan imaging lens, wherein the steering the light beam towards a targetlocation is performed by the illumination lens, and wherein the steeringa reflected light beam that is reflected off of the underlying surfacetowards an image sensor is performed by the imaging lens. In certainembodiments, the reflected light substantially overfills the first setof pixels and not the second set of pixels while the input device isoperating on the underlying surface, and wherein the reflected lightsubstantially overfills the second set of pixels when the input deviceis lifted or tilted off of the underlying surface.

In certain embodiments, a system for operating an input devicecomprises: one or more processors; and one or more machine-readable,non-transitory storage mediums that include instructions configured tocause the one or more processors to perform operations including:generating a light beam by a light source module controlled by one ormore processors of the input device; steering the light beam towards atarget location, wherein the target location corresponds to a spot on anunderlying surface while the input device is operating on the underlyingsurface; steering a reflected light beam that is reflected off of theunderlying surface towards an image sensor of the input device;receiving the reflected light beam by the image sensor, the image sensorcontrolled by the one or more processors; generate tracking data by theimage sensor that corresponds to a two-dimensional (2D) movement of theinput device with respect to the underlying surface based on thereceived reflected light beam; determine that the input device isoperating on and in contact with the underlying surface when thereflected light beam received by the image sensor is located on a firstset of pixels of a plurality of pixels of the image sensor; anddetermine that the input device is operating above and not in contactwith the underlying surface when the reflected light beam received bythe image sensor is located on a second set of pixels of the pluralityof pixels of the image sensor. In some embodiments, the first set ofpixels forms a square shape that receives the reflected light from thelight source module when the input device is operating on the underlyingsurface, wherein the second set of pixels is adjacent to the first setof pixels such that the first set of pixels and the second set of pixelstogether form a rectangle, and wherein the second set of pixels isconfigured at a location relative to the first set of pixels such thatthe reflected light from the light source module moves from the firstset of pixels to the second set of pixels as the input device is liftedoff of the underlying surface. In some embodiments, the instructions arefurther configured to cause the one or more processors to performoperations including: determining a surface type of the underlyingsurface; in response to determining that the surface type is a highcontrast surface, utilizing both the first and second set of pixels fortracking the location of the reflected light beam; and in response todetermining that the surface type is a low contrast surface, utilizingonly the first set of pixels for tracking the location of the reflectedlight beam.

This summary is not intended to identify key or essential features ofthe claimed subject matter, nor is it intended to be used in isolationto determine the scope of the claimed subject matter. The subject mattershould be understood by reference to appropriate portions of the entirespecification of this disclosure, any or all drawings, and each claim.

The foregoing, together with other features and examples, will bedescribed in more detail below in the following specification, claims,and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The features of the various embodiments described above, as well asother features and advantages of certain embodiments of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 shows a simplified diagram of a computer system, according tocertain embodiments.

FIG. 2 shows a simplified block diagram of a system to operate an inputdevice, according to certain embodiments.

FIG. 3 shows a simplified block diagram of a system to operate a hostcomputing device, according to certain embodiments.

FIG. 4A shows aspects of an input device, according to certainembodiments.

FIG. 4B shows aspects of a bottom portion of input device, according tocertain embodiments.

FIG. 5 shows a simplified block diagram of an image sensor circuit,according to certain embodiments.

FIG. 6 shows a simplified diagram of aspects of a movement trackingsystem for an input device, according to certain embodiments.

FIG. 7 shows a pixel array for a conventional image sensor circuit.

FIG. 8 shows an improve pixel array for an image sensor circuit,according to certain embodiments.

FIG. 9 shows aspects of lift detection for an input device, according tocertain embodiments.

FIG. 10 shows a simplified block diagram of a method for performing liftdetection for an input device, according to certain embodiments.

The detailed description is set forth with reference to the accompanyingfigures. Throughout the drawings, it should be noted that like referencenumbers are typically used to depict the same or similar elements,features, and structures.

DETAILED DESCRIPTION

Aspects of the present disclosure relate generally to computerperipheral devices, and more particularly to lift detection with inputdevices, according to certain embodiments.

In the following description, various examples of lift detection withinput devices are described. For purposes of explanation, specificconfigurations and details are set forth in order to provide a thoroughunderstanding of the embodiments. However, it will be apparent to oneskilled in the art that certain embodiments may be practiced orimplemented without every detail disclosed. Furthermore, well-knownfeatures may be omitted or simplified in order to prevent anyobfuscation of the novel features described herein.

The following high level summary is intended to provide a basicunderstanding of some of the novel innovations depicted in the figuresand presented in the corresponding description provided below. Overyears of development and improvement, image sensors in input devices(e.g., computer mice) with optical tracking systems have becomeincreasingly more efficient. In the early 2000s, optical trackingsystems typically employed red LEDs configured at ˜25° (with respect tothe surface) with diffuse optical emissions that allowed for avertically positioned image sensor, however surface coverage (e.g.,surfaces that allowed for reliable detection) was often limited tostructured surfaces that, when illuminated by the LED, produced anilluminated spot with shadows that the sensor is able to capture andcorrelated between successive images. In such systems, the low-angle LEDbeam typically produces a large spot on the underlying surface (e.g.,mouse pad) that the image sensor can reliably detect, even when theinput device is lifted by a small amount. This ability to detectmovement of the input device while it is lifted is called lift-offdetection (LoD). For low-angle, diffuse-emission LEDs, lift-offdetection was often possible at up to 3-4 mm. Sensor cutoff (where thesensor can no longer track movement of the input device while lifted)was typically related to a loss of contrast due to sensor featuresblurring, and diffuse-emission LED systems were highly surface dependentas noted above, with reliable tracking on very structured surfaces(e.g., mouse pads) and unreliable tracking or complete inoperability onmyriad non-structured surfaces (e.g., glass tables, smooth surfaces,etc.) that did not create detectable shadows, as would be appreciated byone of ordinary skill in the art with the benefit of this disclosure.

As tracking technology developed, some input devices employed laserswith highly collimated light with specular optical trackingconfigurations with an optical sensor arranged along the specularreflection angle for movement tracking rather than the earliervertically positioned designs that took advantage of the diffuse natureof early LED emission patterns. The highly collimated spot created by alaser can be significantly smaller than early LED-based spot and cancapture more surface details of the underlying surface rather thansimply generating shadows that the image sensor can detect andcorrelate. As a result, one advantage of laser-based systems is theability to track on myriad surface types including smooth surfaces and,in some cases, glass surfaces. Power efficiency in laser-based systemscan be improved over its LED-based predecessors due to the comparativelysmall spot and low current source requirements used to the drive thelaser. Also, given the small size of the spot compared to the sensorarray that it is trained on during use, the spot may remain within thesensor array as the input device is lifted with LoD thresholds thatoften range from 3-5 mm.

In modern devices, laser-based sensing gave way to improved LED-basedsystems with specular configurations (e.g., made possible bysend/receive lensing apparatuses) that further broaden surface typeusage and further improves power efficiency. Eventually, infra-red (IR)LEDs have been used in specular emission LED systems for a variety ofreasons including further improved power efficiency, low forward voltagecharacteristics (e.g., 1.45 V), tracking coverage, and cost. Onedrawback associated with the improved LED-based systems is that althoughthe specular configuration produces a much smaller spot than LED-basedsystems with diffuse emission patterns (however still extractsinformation of the underlying surface due to the nature of the specularreflection), the spot can be much larger than the laser-basedcounterparts such that the spot can move off of the image sensor whenthe input device is lifted more quickly, resulting in lower LoDthresholds (e.g., 2-3 mm), which could be highly dependent on underlyingsurface characteristics. In 2014, Logitech® pioneered furtherimprovements in the LED-based, specular emission systems that furtherimproved power consumption and employed a surface tuning feature thatpermitted a user to adjust the LoD for specific surface types (e.g.,mouse pads) that could be as low as 1 mm, which was popular in thee-sports community. Some contemporary designs include improved lensesthat further reduce the size of the spot and improve power efficiencywith LoD ranging from 0.8-1.1 mm. Although a low LoD may be preferredfor a number of operational reasons (e.g., improved power efficiency,tracking, etc.), many gamers became accustomed to diffuse-type LEDsystems and often used input device manipulation techniques thatincluded slight lifting (e.g., “skating”) and/or tilting of the inputdevice several millimeters while in use rather than keeping the inputdevice well in contact with the underlying surface, which did not haveparticularly strong deleterious effects on the diffuse-type LED systemswith relatively the high LoD thresholds. However, when skating andtilting movements are applied to specular-type LED systems with lowerLoD thresholds, deleterious effects such as parasitic tracking phenomenamay be introduced (e.g., resulting in spurious cursor tracking) that canbe hard to anticipate and/or correct when the input device is not wellin contact with an underlying surface. This problem affects allcontemporary IR LED specular-based input devices (e.g., computer mice)introduced since 2014, and no effective solutions have been introducedbefore the various embodiments presented in the present disclosure.Thus, aspects of the present invention are directed to enabling a widerworking range (e.g., greater LoD range) in combination with a spotposition detection capability, detecting characteristics of anunderlying surface, or a combination thereof, in a manner that istransparent to the user and with no user-discernable degradation intracking quality or reliability.

In contemporary designs (e.g., high end gaming mice), sensor sizes haveremained relatively large to allow for higher resolution and high speedand acceleration tracking. Typically, the reflected light spot fills theimage sensor, which is ubiquitously a square shaped sensor comprised ofa plurality of pixels in conventional input devices, such that the spotilluminates all (overfills) or nearly all of the pixels in the imagesensor as the input device operates under typical modes of operation(e.g., tracking two-dimensional (2D) movement of the input devicerelative to an underlying surface). Utilizing as many pixels as possibleon the image sensor allows for improved tracking accuracy, resolution,and efficiency, as would be appreciated by one of ordinary skill in theart with the benefit of this disclosure. Unused pixels that do notreceive reflected light during normal operation of the input device canneedlessly waste power in conventional devices, thus the spot (usually acircle) is typically configured to “overfill” the image sensor(historically a square) to ensure that most of the pixels areilluminated and utilized. A typical size ratio of the spot to theplurality of pixels is 1.2:1 (e.g., 120%) in contemporary computer mice.

Although power efficiency and performance over varying surfaces hassubstantially improved over the years with newer specular-type LEDsystems, one negative consequence is that image sensor-based liftdetection accuracy has reduced, as noted above. That is, the practice ofusing an image sensor configured for 2D surface tracking (e.g., along anX-Y plane) to also detect when the input device is lifted off of thesurface (e.g., tracking a “Z-height”) has gotten progressively worse. Asdescribed above, in some earlier diffuse-based LED systems, the spot maynot have completely filled the square array (e.g., in some designs, thespot only filled ¼ of the pixel array), so that when the input devicewas lifted, the spot proportionally moved along the image sensor butstill illuminated a number of the pixels until the input device waslifted and/or tilted to a height (threshold) where the spot moved off ofthe image sensor and movement tracking was no longer possible. Inpractical applications, a user could lift the computer mouse, move it,and place it back down (e.g., moving the input device from the edge of amouse pad and back to a center point, known colloquially as “skating”)and still accurately track the X-Y movement relative to the underlyingsurface, even when the input device was lifted up to a threshold heightbecause the reflected light spot was still illuminating pixels on theimage sensor. Because modern specular-based input devices substantiallyoverfill the image sensor such that a very small amount of lift willcause the reflected spot to move off or mostly off of the image sensor,the input device will lose X-Y tracking accuracy, which can manifest asspurious tracking (e.g., where a computer mouse-controlled cursorappears to randomly jump to different locations), introduce inaccuratemovement artifacts (e.g., jitter), or the like. These types ofdeleterious tracking conditions are considered unacceptable in moderninput devices, so contemporary designs often employ a secondary trackingsystem design specifically for lift detection (e.g., only trackingZ-height) to address these performance issues.

Aspects of the invention solve this problem by extending the pixel arrayto have a nominal position (e.g., a square array of pixels 820) wherethe spot (890) illuminates the pixels (e.g., overfilling pixels 820)while the input device operates on and along an underlying surface(e.g., as shown in FIG. 8), and an extended position (e.g., pixels 830)that the spot transitions to (traverses to) as the input device islifted off of the underlying surface (e.g., as shown in FIG. 9). Thus,the same sensor array can be used for accurate and efficient 2D trackingalong an underlying surface and, using the same sensor array, accuratelift detection and tracking in a lifted state can be performed withoutrequiring any additional and independent lift detection systems, asfound in other contemporary systems. That is, the spot moves along theimage sensor array from the first set of pixels 820 to the second set ofpixels 830 of the same array 810, thereby extending a tracking computewindow (e.g., tracking a spot centroid), as further described in thedescription below, and increasing the working range of the input device(e.g., allowing a user to skate the input device with substantiallyreduced or eliminated deleterious effects). As described above, imagesensors have historically been square-shaped to optimally match thespot. The rectangular shape of the image sensor (e.g., as shown anddescribed below with respect to FIGS. 8-9) has not been used incontemporary or historical designs, as during normal circumstances whilethe input device is moving on an underlying surface, the extendedportion of the image sensor (830) would not be utilized, which wouldresult in reduced power efficiency, a greater circuit footprint, andhigher manufacturing costs. Contemporary solutions to this issuetypically involve an additional second sensor system dedicated to liftdetection. Consequently, such designs may detect when a lift conditionoccurs, but they do not allow for any change or improvement in 2Dtracking, which may add a limited benefit of causing the input device toknow when to stop tracking to prevent spurious detection. Thus, addingmore pixels to extend the square-shaped image sensor in a direction totrack the spot as it traverses along the pixel array would not be anobvious solution for the issue of lift detection for the variousefficiency reasons given above and as evidenced by the fact that arectangular image sensor configured in this manner has never been usedbefore, and such new extended arrays provide the substantial benefit ofallowing good tracking accuracy at increased z-heights on a singlesystem to increase a working range of the input device, which includes2D tracking in a lifted state to accommodate common practices likeskating, tilting, and the like, with reduced or eliminated deleterioustracking effects, as noted above. Furthermore, aspects of the inventioncan utilize the extended array to not only determine when a lift or tiltcondition occurs, but can be programmed to set a LoD threshold that canallow a gamer to customize how the input device operates when lifted.

It is to be understood that this high level summary is presented toprovide the reader with a baseline understanding of some of the novelaspects of the present disclosure and a roadmap to the details thatfollow. This high level summary in no way limits the scope of thevarious embodiments described throughout the detailed description andeach of the figures referenced above are further described below ingreater detail and in their proper scope.

Typical System Environment

FIG. 1 shows a simplified diagram of a computer system 100, according tocertain embodiments. Computer system 100 can include computer 110,monitor 120, input device 130, and keyboard 140. In some embodiments,input device 130 can be a computer mouse, a remote control device, agame controller (e.g., game pad, joystick, etc.), a smart phone, orother suitable device that can be used to convert analog inputs intodigital signals for computer processing. For computer system 100, inputdevice 130 can be configured to control various aspects of computer 110and monitor 120.

Although the host computing device is shown as a laptop computer, othertypes of host computing devices can be used including gaming systems,desktop computers, set top boxes, entertainment systems, a tablet or“phablet” computer, or any other suitable host computing device (e.g.,smart phone, smart wearable, or the like). In some cases, multiple hostcomputing devices may be used and one or more of the peripheral devicesmay be each communicatively coupled to one or more of the host computingdevices (e.g., a mouse may be coupled to multiple host computingdevices). A host computing device may be referred to herein as a “hostcomputer,” “host device,” “host computing device,” “computing device,”“computer,” or the like, and may include a machine readable medium (notshown) configured to store computer code, such as driver software,firmware, and the like, where the computer code may be executable by oneor more processors of the host computing device(s) to control aspects ofthe host computing device via the one or more peripheral input devices.

A typical peripheral device can include any suitable input peripheraldevice, output peripheral device or input/output peripheral deviceincluding those shown (e.g., a computer mouse) and not shown (e.g., gamecontroller, remote control, wearables (e.g., gloves, watch, head mounteddisplay), AR/VR controller, stylus device, gaming pedals/shifters, orother suitable device) that can be used to convert analog inputs intodigital signals for computer processing. In some embodiments, computerperipheral device 130 can be configured to provide control signals formovement tracking (e.g., x-y movement on a planar surface,three-dimensional “in-air” movements, etc.), touch and/or gesturedetection, lift detection, orientation detection (e.g., in 3degrees-of-freedom (DOF) system, 6 DOF systems, etc.), power managementcapabilities, input detection (e.g., buttons, scroll wheels, etc.),output functions (e.g., LED control, haptic feedback, etc.), or any ofmyriad other features that can be provided by a computer peripheraldevice, as would be appreciated by one of ordinary skill in the art.

A computer peripheral device may be referred to as an “input device,”“peripheral input device,” “peripheral,” or the like. The majority ofthe embodiments described herein generally refer to computer peripheraldevice 130 as a computer mouse or similar input device, however itshould be understood that computer peripheral device 130 can be anysuitable input/output (I/O) device (e.g., user interface device, controldevice, input unit, or the like) that may be adapted to utilize thenovel embodiments described and contemplated herein.

Typical System Embodiment for Operating an Input Device

FIG. 2 shows a simplified block diagram of a system 200 to operate inputdevice 130, according to certain embodiments. System 200 may includeprocessor(s) 210, input detection block 220, movement tracking block230, power management block 240, and communication block 250. Each ofsystem blocks 220-250 can be in electrical communication with processor210. System 200 may further include additional systems that are notshown or described to prevent obfuscation of the novel featuresdescribed herein. System blocks 220-250 (also referred to as “modules”)may be implemented as separate modules, or alternatively, more than onesystem block may be implemented in a single module. In the contextdescribed herein, system 200 can be incorporated into any computerperipheral device described herein and may be configured to perform anyof the various methods of lift detection as described below at leastwith respect to FIGS. 5-10, as would be appreciated by one of ordinaryskill in the art with the benefit of this disclosure.

In certain embodiments, processor(s) 210 may include one or moremicroprocessors and can be configured to control the operation of system200. Alternatively or additionally, processor(s) 210 may include one ormore microcontrollers (MCUs), digital signal processors (DSPs), or thelike, with supporting hardware and/or firmware (e.g., memory,programmable I/Os, etc.), and/or software, as would be appreciated byone of ordinary skill in the art. Processor(s) 210 can control some orall aspects of the operation of computer peripheral device 130 (e.g.,system blocks 220-250). Alternatively or additionally, some of systemblocks 220-250 may include an additional dedicated processor, which maywork in conjunction with processor(s) 210. For instance, MCUs, μCs,DSPs, and the like, may be configured in other system blocks of system200. Communications block 250 may include a local processor, forinstance, to control aspects of communication with computer 110 (e.g.,via Bluetooth, Bluetooth LE, RF, IR, hardwire, ZigBee, Z-Wave, LogitechUnifying, or other communication protocol). Processor(s) 210 may belocal to the peripheral device (e.g., contained therein), may beexternal to the peripheral device (e.g., off-board processing, such asby a corresponding host computing device), or a combination thereof.Processor(s) 210 may perform any of the various functions and methods(e.g., method 1000) described and/or covered by this disclosure inconjunction with any other system blocks in system 200. In someimplementations, processor 302 of FIG. 3 may work in conjunction withprocessor 210 to perform some or all of the various methods describedthroughout this disclosure. In some embodiments, multiple processors mayenable increased performance characteristics in system 200 (e.g., speedand bandwidth), however multiple processors are not required, nornecessarily germane to the novelty of the embodiments described herein.One of ordinary skill in the art would understand the many variations,modifications, and alternative embodiments that are possible.

Input detection module 250 can control the detection of auser-interaction with input elements (also referred to as “inputmembers”) on computer peripheral device 150. Input detection block 220can detect user inputs from motion sensors, keys, buttons, rollerwheels, scroll wheels, track balls, touch pads (e.g., one and/ortwo-dimensional touch sensitive touch pads), click wheels, dials,keypads, microphones, GUIs, touch-sensitive GUIs, image sensor baseddetection such as gesture detection (e.g., via webcam), audio baseddetection such as voice input (e.g., via microphone), or the like, aswould be appreciated by one of ordinary skill in the art with thebenefit of this disclosure. Alternatively, the functions of inputdetection block 220 can be subsumed by processor 210, or in combinationtherewith.

In some embodiments, input detection block 220 can detect a touch ortouch gesture on one or more touch sensitive surfaces on input device130. Input detection block 220 can include one or more touch sensitivesurfaces or touch sensors. Touch sensors generally comprise sensingelements suitable to detect a signal such as direct contact,electromagnetic or electrostatic fields, or a beam of electromagneticradiation. Touch sensors can typically detect changes in a receivedsignal, the presence of a signal, or the absence of a signal. A touchsensor may include a source for emitting the detected signal, or thesignal may be generated by a secondary source. Touch sensors may beconfigured to detect the presence of an object at a distance from areference zone or point (e.g., <5 mm), contact with a reference zone orpoint, or a combination thereof. Certain embodiments of input device 130may or may not utilize touch detection or touch sensing capabilities.

Input detection block 220 can include touch and/or proximity sensingcapabilities. Some examples of the types of touch/proximity sensors mayinclude, but are not limited to, resistive sensors (e.g., standardair-gap 4-wire based, based on carbon loaded plastics which havedifferent electrical characteristics depending on the pressure (FSR),interpolated FSR, etc.), capacitive sensors (e.g., surface capacitance,self-capacitance, mutual capacitance, etc.), optical sensors (e.g.,infrared light barriers matrix, laser based diode coupled withphoto-detectors that could measure the time of flight of the light path,etc.), acoustic sensors (e.g., piezo-buzzer coupled with microphones todetect the modification of a wave propagation pattern related to touchpoints, etc.), or the like.

In some embodiments, input detection block 220 may also control someoutput functions of input device 130, such as a number of visual outputelements (e.g., mouse cursor, LEDs, LCDs), displays, audio outputs(e.g., speakers), haptic output systems, or the like. One of ordinaryskill in the art with the benefit of this disclosure would appreciatethe many modifications, variations, and alternative embodiments thereof.

Movement tracking block 230 can be configured to track a movement ofinput device 130. Movement tracking block 230 can use optical sensorsystems that utilize a light-emitting diode(s) (LEDs) and an imagingarray of photodiodes (referred to individual as “pixels” andcollectively as a “pixel array”) to detect a movement of input device130 relative to an underlying surface. Input device 130 may optionallyinclude movement tracking hardware that utilizes coherent (laser) light.In certain embodiments, an optical sensor is disposed on the bottom sideof input device 130, as shown in FIG. 4B. Movement tracking block 230can provide positional data (e.g., X-Y coordinate data) and liftdetection data. For example, an optical sensor can detect when a userlifts input device 130 off of a underlying surface and can send thatdata to processor 210 for further processing. In some embodiments,processor 210, movement tracking block 230 (which may include anadditional dedicated processor), or a combination thereof may performsome or all of the novel functions described herein including processingdifferent set of pixels of a plurality of pixels of an image sensor todetermine when the input device is lifted off of an underlying surface,as described below at least with respect to FIGS. 8-10.

In certain embodiments, an inertial measurement unit (IMU) can be usedto both movement and lift detection. An IMU may incorporate one or moreaccelerometers and/or gyroscopes, among other devices for movementdetection. Accelerometers can be electromechanical devices (e.g.,micro-electromechanical systems (MEMS) devices) configured to measureacceleration forces (e.g., static and dynamic forces). One or moreaccelerometers can be used to detect three dimensional (3D) positioning.For example, 3D tracking can utilize a three-axis accelerometer or twotwo-axis accelerometers (e.g., in a “3D air mouse.” As noted above andfurther described below, accelerometers can further determine if inputdevice 130 has been lifted off of a surface and provide movement datathat may include the velocity, physical orientation, and acceleration ofinput device 130. In some embodiments, gyroscope(s) can be used in lieuof or in conjunction with accelerometer(s) to determine movement orinput device orientation.

Power management system 230 can be configured to manage powerdistribution, recharging, power efficiency, haptic motor power control,and the like. In some embodiments, power management system 230 caninclude a battery (not shown), a Universal Serial Bus (USB)-basedrecharging system for the battery (not shown), and power managementdevices (e.g., voltage regulators—not shown), and a power grid withinsystem 200 to provide power to each subsystem (e.g., communicationsblock 240, etc.). In certain embodiments, the functions provided bypower management system 230 may be incorporated into processor(s) 210.Alternatively, some embodiments may not include a dedicated powermanagement block. For example, functional aspects of power managementblock 240 may be subsumed by another block (e.g., processor(s) 210) orin combination therewith. The power source can be a replaceable battery,a rechargeable energy storage device (e.g., super capacitor, LithiumPolymer Battery, NiMH, NiCd), or a corded power supply. The rechargingsystem can be an additional cable (specific for the recharging purpose)or it can use a USB connection to recharge the battery.

Communication system 240 can be configured to enable wirelesscommunication with a corresponding host computing device (e.g., 110), orother devices and/or peripherals, according to certain embodiments.Communication system 240 can be configured to provide radio-frequency(RF), Bluetooth®, Logitech proprietary communication protocol (e.g.,Unifying, Gaming Light Speed, or others), infra-red (IR), ZigBee®,Z-Wave, or other suitable communication technology to communicate withother computing devices and/or peripheral devices. System 200 mayoptionally comprise a hardwired connection to the corresponding hostcomputing device. For example, computer peripheral device 130 can beconfigured to receive a USB, FireWire®, Thunderbolt®, or otheruniversal-type cable to enable bi-directional electronic communicationwith the corresponding host computing device or other external devices.Some embodiments may utilize different types of cables or connectionprotocol standards to establish hardwired communication with otherentities. In some aspects, communication ports (e.g., USB), power ports,etc., may be considered as part of other blocks described herein (e.g.,input detection module 120, etc.). In some aspects, communication system240 can send reports generated by the processor(s) 210 (e.g., HID data,streaming or aggregated data, etc.) to a host computing device. In somecases, the reports can be generated by the processor(s) only, inconjunction with the processor(s), or other entity in system 200.Communication system 240 may incorporate one or more antennas,oscillators, etc., and may operate at any suitable frequency band (e.g.,2.4 GHz), etc. One of ordinary skill in the art with the benefit of thisdisclosure would appreciate the many modifications, variations, andalternative embodiments thereof.

Although certain systems may not expressly discussed, they should beconsidered as part of system 200, as would be understood by one ofordinary skill in the art. For example, system 200 may include a bussystem to transfer power and/or data to and from the different systemstherein. In some embodiments, system 200 may include a storage subsystem(not shown). A storage subsystem can store one or more software programsto be executed by processors (e.g., in processor(s) 210). It should beunderstood that “software” can refer to sequences of instructions that,when executed by processing unit(s) (e.g., processors, processingdevices, etc.), cause system 200 to perform certain operations ofsoftware programs. The instructions can be stored as firmware residingin read only memory (ROM) and/or applications stored in media storagethat can be read into memory for processing by processing devices.Software can be implemented as a single program or a collection ofseparate programs and can be stored in non-volatile storage and copiedin whole or in-part to volatile working memory during program execution.From a storage subsystem, processing devices can retrieve programinstructions to execute in order to execute various operations (e.g.,software-controlled spring auto-adjustment, etc.) as described herein.

It should be appreciated that system 200 is meant to be illustrative andthat many variations and modifications are possible, as would beappreciated by one of ordinary skill in the art. System 200 can includeother functions or capabilities that are not specifically described here(e.g., mobile phone, global positioning system (GPS), power management,one or more cameras, various connection ports for connecting externaldevices or accessories, etc.). While system 200 is described withreference to particular blocks (e.g., input detection block 220), it isto be understood that these blocks are defined for understanding certainembodiments of the invention and is not intended to imply thatembodiments are limited to a particular physical arrangement ofcomponent parts. The individual blocks need not correspond to physicallydistinct components. Blocks can be configured to perform variousoperations, e.g., by programming a processor or providing appropriateprocesses, and various blocks may or may not be reconfigurable dependingon how the initial configuration is obtained. Certain embodiments can berealized in a variety of apparatuses including electronic devicesimplemented using any combination of circuitry and software.Furthermore, aspects and/or portions of system 200 may be combined withor operated by other sub-systems as informed by design. For example,power management block 240 and/or movement tracking block 230 may beintegrated with processor(s) 210 instead of functioning as a separateentity.

It should be appreciated that system 200 is illustrative and thatvariations and modifications are possible. System 200 can have othercapabilities not specifically described herein. Further, while system200 is described with reference to particular blocks, it is to beunderstood that these blocks are defined for convenience of descriptionand are not intended to imply a particular physical arrangement ofcomponent parts. Further, the blocks need not correspond to physicallydistinct components. Blocks can be configured to perform variousoperations, e.g., by programming a processor or providing appropriatecontrol circuitry, and various blocks might or might not bereconfigurable depending on how the initial configuration is obtained.

Embodiments of the present invention can be realized in a variety ofapparatuses including electronic devices (e.g., peripheral devices)implemented using any combination of circuitry and software.Furthermore, aspects and/or portions of system 200 may be combined withor operated by other sub-systems as required by design. For example,input detection module 250 and/or memory 220 may operate withinprocessor(s) 210 instead of functioning as a separate entity. Inaddition, the inventive concepts described herein can also be applied toany peripheral device. Further, system 200 can be applied to any of thecomputer peripheral devices described in the embodiments herein, whetherexplicitly, referentially, or tacitly described (e.g., would have beenknown to be applicable to a particular computer peripheral device by oneof ordinary skill in the art). The foregoing embodiments are notintended to be limiting and those of ordinary skill in the art with thebenefit of this disclosure would appreciate the myriad applications andpossibilities.

System for Operating a Host Computing Device

FIG. 3 is a simplified block diagram of a computing device 300,according to certain embodiments. Computing device 300 can implementsome or all functions, behaviors, and/or capabilities described abovethat would use electronic storage or processing, as well as otherfunctions, behaviors, or capabilities not expressly described. Computingdevice 300 includes a processing subsystem (processor(s)) 302, a storagesubsystem 306, user interfaces 314, 316, and a communication interface312. Computing device 300 can also include other components (notexplicitly shown) such as a battery, power controllers, and othercomponents operable to provide various enhanced capabilities. In variousembodiments, computing device 300 can be implemented in a host computingdevice, such as a desktop 110 or laptop computer, mobile device (e.g.,tablet computer, smart phone, mobile phone), wearable device, mediadevice, or the like, in peripheral devices (e.g., keyboards, etc.) incertain implementations.

Processor(s) 302 can include MCU(s), micro-processors, applicationspecific integrated circuits (ASICs), digital signal processors (DSPs),digital signal processing devices (DSPDs), programmable logic devices(PLDs), field programmable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, or electronic units designed toperform a function or combination of methods, functions, etc., describedthroughout this disclosure.

Storage subsystem 306 can be implemented using a local storage and/orremovable storage medium, e.g., using disk, flash memory (e.g., securedigital card, universal serial bus flash drive), or any othernon-transitory storage medium, or a combination of media, and caninclude volatile and/or non-volatile storage media. Local storage caninclude a memory subsystem 308 including random access memory (RAM) 318such as dynamic RAM (DRAM), static RAM (SRAM), synchronous dynamic RAM(e.g., DDR), or battery backed up RAM or read-only memory (ROM) 320, ora file storage subsystem 310 that may include one or more code modules.In some embodiments, storage subsystem 306 can store one or moreapplications and/or operating system programs to be executed byprocessing subsystem 302, including programs to implement some or alloperations described above that would be performed using a computer. Forexample, storage subsystem 306 can store one or more code modules forimplementing one or more method steps described herein.

A firmware and/or software implementation may be implemented withmodules (e.g., procedures, functions, and so on). A machine-readablemedium tangibly embodying instructions may be used in implementingmethodologies described herein. Code modules (e.g., instructions storedin memory) may be implemented within a processor or external to theprocessor. As used herein, the term “memory” refers to a type of longterm, short term, volatile, nonvolatile, or other storage medium and isnot to be limited to any particular type of memory or number of memoriesor type of media upon which memory is stored.

Moreover, the term “storage medium” or “storage device” may representone or more memories for storing data, including read only memory (ROM),RAM, magnetic RAM, core memory, magnetic disk storage mediums, opticalstorage mediums, flash memory devices and/or other machine readablemediums for storing information. The term “machine-readable medium”includes, but is not limited to, portable or fixed storage devices,optical storage devices, wireless channels, and/or various other storagemediums capable of storing instruction(s) and/or data.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages, and/or any combination thereof. When implementedin software, firmware, middleware, scripting language, and/or microcode,program code or code segments to perform tasks may be stored in amachine readable medium such as a storage medium. A code segment (e.g.,code module) or machine-executable instruction may represent aprocedure, a function, a subprogram, a program, a routine, a subroutine,a module, a software package, a script, a class, or a combination ofinstructions, data structures, and/or program statements. A code segmentmay be coupled to another code segment or a hardware circuit by passingand/or receiving information, data, arguments, parameters, and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted by suitable means including memory sharing,message passing, token passing, network transmission, etc. Thesedescriptions of software, firmware, storage mediums, etc., apply tosystems 200 and 300, as well as any other implementations within thewide purview of the present disclosure. In some embodiments, aspects ofthe invention (e.g., surface classification) may be performed bysoftware stored in storage subsystem 306, stored in memory 220 of inputdevice 130, or both. One of ordinary skill in the art with the benefitof this disclosure would appreciate the many modifications, variations,and alternative embodiments thereof.

Implementation of the techniques, blocks, steps and means describedthroughout the present disclosure may be done in various ways. Forexample, these techniques, blocks, steps and means may be implemented inhardware, software, or a combination thereof. For a hardwareimplementation, the processing units may be implemented within one ormore ASICs, DSPs, DSPDs, PLDs, FPGAs, processors, controllers,micro-controllers, microprocessors, other electronic units designed toperform the functions described above, and/or a combination thereof.

Each code module may comprise sets of instructions (codes) embodied on acomputer-readable medium that directs a processor of a computing device110 to perform corresponding actions. The instructions may be configuredto run in sequential order, in parallel (such as under differentprocessing threads), or in a combination thereof. After loading a codemodule on a general purpose computer system, the general purposecomputer is transformed into a special purpose computer system.

Computer programs incorporating various features described herein (e.g.,in one or more code modules) may be encoded and stored on variouscomputer readable storage media. Computer readable media encoded withthe program code may be packaged with a compatible electronic device, orthe program code may be provided separately from electronic devices(e.g., via Internet download or as a separately packaged computerreadable storage medium). Storage subsystem 306 can also storeinformation useful for establishing network connections using thecommunication interface 312.

Computer system 300 may include user interface input devices 314elements (e.g., touch pad, touch screen, scroll wheel, click wheel,dial, button, switch, keypad, microphone, etc.), as well as userinterface output devices 316 (e.g., video screen, indicator lights,speakers, headphone jacks, virtual- or augmented-reality display, etc.),together with supporting electronics (e.g., digital to analog or analogto digital converters, signal processors, etc.). A user can operateinput devices of user interface 314 to invoke the functionality ofcomputing device 300 and can view and/or hear output from computingdevice 300 via output devices of user interface 316.

Processing subsystem 302 can be implemented as one or more processors(e.g., integrated circuits, one or more single core or multi coremicroprocessors, microcontrollers, central processing unit, graphicsprocessing unit, etc.). In operation, processing subsystem 302 cancontrol the operation of computing device 300. In some embodiments,processing subsystem 302 can execute a variety of programs in responseto program code and can maintain multiple concurrently executingprograms or processes. At a given time, some or all of a program code tobe executed can reside in processing subsystem 302 and/or in storagemedia, such as storage subsystem 304. Through programming, processingsubsystem 302 can provide various functionality for computing device300. Processing subsystem 302 can also execute other programs to controlother functions of computing device 300, including programs that may bestored in storage subsystem 304.

Communication interface (also referred to as network interface) 312 canprovide voice and/or data communication capability for computing device300. In some embodiments, communication interface 312 can include radiofrequency (RF) transceiver components for accessing wireless datanetworks (e.g., Wi-Fi network; 3G, 4G/LTE; etc.), mobile communicationtechnologies, components for short range wireless communication (e.g.,using Bluetooth communication standards, NFC, etc.), other components,or combinations of technologies. In some embodiments, communicationinterface 312 can provide wired connectivity (e.g., universal serial bus(USB), Ethernet, universal asynchronous receiver/transmitter, etc.) inaddition to, or in lieu of, a wireless interface. Communicationinterface 312 can be implemented using a combination of hardware (e.g.,driver circuits, antennas, modulators/demodulators, encoders/decoders,and other analog and/or digital signal processing circuits) and softwarecomponents. In some embodiments, communication interface 312 can supportmultiple communication channels concurrently.

User interface input devices 314 may include any suitable computerperipheral device (e.g., computer mouse 130, keyboard, gamingcontroller, remote control, stylus device, etc.), as would beappreciated by one of ordinary skill in the art with the benefit of thisdisclosure. User interface output devices 316 can include displaydevices (e.g., a monitor, television, projection device, etc.), audiodevices (e.g., speakers, microphones), haptic devices, etc. Note thatuser interface input and output devices are shown to be a part of system300 as an integrated system. In some cases, such as in laptop computers,this may be the case as keyboards and input elements as well as adisplay and output elements are integrated on the same host computingdevice. In some cases, the input and output devices may be separate fromsystem 300, as shown in FIG. 1. One of ordinary skill in the art withthe benefit of this disclosure would appreciate the many modifications,variations, and alternative embodiments thereof.

It will be appreciated that computing device 300 is illustrative andthat variations and modifications are possible. A host computing devicecan have various functionality not specifically described (e.g., voicecommunication via cellular telephone networks) and can includecomponents appropriate to such functionality. While the computing device300 is described with reference to particular blocks, it is to beunderstood that these blocks are defined for convenience of descriptionand are not intended to imply a particular physical arrangement ofcomponent parts. For example, processing subsystem 302, storagesubsystem 306, user interfaces 314, 316, and communications interface312 can be in one device or distributed among multiple devices. Further,the blocks need not correspond to physically distinct components. Blockscan be configured to perform various operations, e.g., by programming aprocessor or providing appropriate control circuitry, and various blocksmight or might not be reconfigurable depending on how an initialconfiguration is obtained. Embodiments of the present invention can berealized in a variety of apparatus including electronic devicesimplemented using a combination of circuitry and software. Hostcomputing devices or even peripheral devices described herein can beimplemented using system 300.

Various Features for Certain Embodiments of an Input Device

FIG. 4A shows aspects of a computer peripheral device 400, according tocertain embodiments. Computer peripheral device 400 can include housing410 (e.g., the “shell,” “chassis,” or “body” of the computer peripheraldevice), left button 420, right button 430, scroll wheel 440 and buttons450, 460, as well as any other suitable input elements (e.g., additionalbuttons, side scroll wheels, touch sensors, etc.) or output elements(e.g., light emitting diodes (LEDs), displays, haptic feedback elements,speakers, etc.), and the like. In some cases, button 450 may be a modeselection button. For example, button 450 may be depressed to manuallyindicate that the computer peripheral device is being used on adifferent surface type. For instance, depressing button 450 may cyclethrough a series of surface types including gaming mouse pad, standardmouse pad, wood surface, metal surface, glass surface, etc., as furtherdescribed in U.S. patent application Ser. No. 16/913,391, filed on Jun.26, 2020, which is incorporated by reference into this application inits entirety for all purposes. Alternatively or additionally, othermodes of operation are possible with different performancecharacteristics, as would be understood by one of ordinary skill in theart. Input device 400 may be input device 130 of FIG. 1.

In some embodiments, buttons 450, 460 may be configured to switchcommunication between host computing devices. For instance, someembodiments may have multi-host connectivity such that computerperipheral device 400 may communication with a first host computer(e.g., a PC laptop) and switch to a second host computer (e.g., a Maccomputer) in response to a corresponding button press, as furtherdescribed in U.S. patent application Ser. No. 14/884,381, which isincorporated by reference into this application in its entirety for allpurposes. Alternatively or additionally, switching between hosts may beachieved by, for example, moving a corresponding cursor to an edge of adisplay in a “flow” enabled system, as further described in U.S. patentapplication Ser. No. 15/226,770, which is incorporated by reference intothis application in its entirety for all purposes. Buttons 450, 460 orany other computer peripheral devices can be configured in any suitablemanner and may utilize any suitable function, which can be pre-set oruser programmed (e.g., via corresponding driver software on a hostcomputing device), as would be understood by one of ordinary skill inthe art.

FIG. 4B shows aspects of a bottom portion of computer peripheral device400, according to certain embodiments. The bottom of computer peripheraldevice 400 can include one or more feet 470, an image sensor 480 (e.g.,CMOS sensor using an IR LED lamp), and a power switch 485. Additionalinput elements (e.g., buttons, sliders, etc.) may be included. In somecases, power switch 485 may be located elsewhere on the mouse or may notbe included at all (e.g., computer peripheral device 400 may powerup/power down based on usage). Button 495 may be a mode selection switch(e.g., switch for selecting a first mode of operation or a second modeof operation), a multi-host computer selection button, or the like. Insome embodiments, button 495 may be a communication protocol selectionbutton. For example, pressing button 495 may switch between aproprietary high-frame rate communication protocol or a lower powerlower frame rate communication protocol (e.g., Bluetooth® LE). One ofordinary skill in the art with the benefit of this disclosure wouldunderstand the many variations, modifications, and alternativeembodiments thereof.

In some embodiments, image sensor 480 is typically located near thecenter of the bottom portion of computer peripheral device 400, asshown. Image sensor 480 can be a single sensor, but can operate in oneor multiple modes of operation (e.g., surface tracking, changingoperating parameters to adapt to particular surface types andcorresponding surface classifications, as further described below), liftand/or tilt detection, and the like, according to certain embodiments.An image sensor can be a complementary metal-oxide semiconductor (CMOS)sensor that captures images of the underlying surface and sends eachimage to a processor (e.g., processor 210, on-board processing on thesensor, etc., to perform image correlation and displacementcalculations, etc.) for analysis. Other types of image sensors may beused, including charge-coupled devices (CCD), N-typemetal-oxide-semiconductors (NMOS), hybrid devices (e.g., CCD/CMOS), orthe like, as would be understood by one of ordinary skill in the art.The processor can detect patterns in the images and see how thosepatterns have moved since the previous image, and based on changes inthe patterns over a sequence of images, the processor can determine howfar and what direction the corresponding computer peripheral device hasmoved, which can be sent to the host computer to control one or morefunctions (e.g., control a cursor on a display, control an audio volumein a music application, etc.). This process can occur many hundreds ifnot thousands of times per second to accurately detect movement of alltypes including a range of movement speeds and accelerations. Typically,the image sensor is overfilled by a light source (e.g., IR LED) toutilize as many pixels as possible to achieve good resolution andaccuracy. In some embodiments, like that shown in FIG. 8, a first set ofpixels (820) of the image sensor (810) is illuminated such that they areoverfilled to operate as noted above, and a second set of pixels (830)or a majority thereof are not illuminated and are present to detect whenthe illumination spot moves due to the input device being tilted orlifted. When such an action occurs, the illumination spot can move tothe second set of pixels, which can be used to determine an amount oflift of the input device, as further described below.

To illustrate some basic operational fundamentals of opticalsensor-based computer peripheral devices (e.g., input device 130, 400),frame rates and memory slots are briefly described here, as they aresome of the performance characteristics (among others) of a computerperipheral device that can be adjusted and optimized for a particularclassified surface type, as further described below. In an opticalsensor-based computer peripheral device, a “frame rate” can define afrequency at which the image sensor takes images of an underlyingsurface.

Generally, quick movements (e.g., 20 ips or more—typical in acompetitive gaming setting) with the computer peripheral device maypreferably be detected using a fast frame rate (e.g., 5 kHz or more) tofully capture the movement with accuracy (e.g., how close themeasurement is to the actual movement speed and/or acceleration) andprecision (e.g., how repeatable an identical measurement is). Likewise,slow movements (e.g., 1-5 ips—typical with productivity software) withthe computer peripheral device may be adequately detected with a slowerframe rate (e.g., 1 kHz), while still achieving accuracy and precision.Higher frame rates tend to cause the input device (e.g., system 200) toconsume more power than do lower frame rates. In some cases, surfaceconditions can also affect power consumption. For example, surfaces witha high density of surface features (e.g., a gaming mouse pad) may beeasier to track movement on as compared to surfaces with few surfacefeatures because there are more points of reference for detectingmovement. Thus, a computer peripheral device operating on a surface witha low density of surface features (e.g., glass, monochromatic metalsurfaces, etc.) may use more light intensity and/or a higher frame ratefor a particular movement and/or acceleration than the computerperipheral device operating on a surface with a high density of surfacefeatures under the same movement and acceleration conditions.

In certain embodiments, a number of memory slots may be used tocorrelate movement of the input device with respect to the underlyingsurface. Memory slots can be used to store images taken by a pixel arrayin an optical sensor. Input device 400 can use a number of memory slotsto save successive image sensor images that are used to detect movementof input device 400 along an underlying surface (e.g., using inputdetection module 250). At minimum, two memory slots are needed tocorrelate movement. For instance, a first page (saved to a first memoryslot) may include a surface feature or particle and a second page (savedto a second memory slot) may include the same surface feature orparticle, but captured at a difference time wherein, if input device 400is moved, the same surface feature or particle will be located adistance from the position shown in the first page. Note that a “page”can be referred to as an “image” for purposes of this disclosure. Thedetected difference of location is used to interpolate a movement of theinput device with respect to the underlying surface, as would beunderstood by one of ordinary skill in the art. “Memory slots” may beinterchangeably referred to as “memory blocks,” (not to be confused withmemory “block” 220) “memory pages,” “memory cells,” and the like. Thememory slots may be part of and/or controlled by processor 210, inputdetection module 250, or a combination thereof. In some cases, memoryslots may be stored on external memory (e.g., external to processor 210and/or movement tracking block 230) and controlled by one or moreresources of system 200. In certain embodiments, the memory slots arestored on the image sensor silicon and may be controlled by image sensor480, processor 210, or a combination thereof. In some cases, the imagesensor can be subsumed, wholly or in part, by input detection module220. One of ordinary skill in the art would understand the manyvariations, modifications, and alternative embodiments thereof.

A Typical Image Sensor Architecture According to Certain Embodiments

FIG. 5 shows a simplified block diagram of an image sensor circuit 500,according to certain embodiments. Image sensor circuit 500 may beincluded in system 200 as part of input detection block 220, processor210, a combination thereof, as its own entity, or the like. Image sensorcircuit 500 can perform aspects of 2D tracking on an underlying surfaceand lift detection of input device 400, as described herein and mayinclude an analog front end 510, a voltage regulator block 520, ananalog-to-digital (A/D) converter 530, a logic back end 540, aninterface block 550, an oscillator circuit 560, and memory block 570,among other features.

Analog front end 510 may include an extended pixel array 810 comprisinga plurality of pixels, as shown in FIG. 8. The pixel array can beconfigured and aligned such that an illumination spot is aligned on afirst set of pixels 820 of the plurality of pixels during normaloperation where the input device is operating on and along an underlyingsurface, and moves to a second set of pixels 830 as the input device islifted, where the illumination spot is sourced by a light source thatprojects light that is bounced off of the underlying surface, as furtherdescribed below. Voltage regulator block 520 can include various systemand methods for performing voltage regulation on image sensor circuit500. A/D converter 530 can be configured to convert analog signals,generated by the plurality pixels in response to photons striking theirsurfaces, into digital signals that are output to logic backend 540 forfurther processing. Although not shown, A/D converter 530 can include anumber of system blocks including multiplexors and comparators toaddress (multiplex/demultiplex) and convert each analog input (e.g.,successive images taken by the analog front end (e.g., pixel array 810)into a digital signal for processing by logic backend 540 (or processor210, or a combination thereof), as would be understood by one ofordinary skill in the art with the benefit of this disclosure. Based onthe individual currents from each pixel, the border (edge) of theilluminated spot where a threshold current value is reached, etc., canbe determined and the corresponding centroid of the spot and Z height ofthe input device can be calculated by logic backend 540, as describedbelow. Typically, the pixel current pulse is proportional in magnitudeto its corresponding illumination. Oscillator block 560 generates one ormore reference and/or driver signals for image sensor circuit 500.Memory block 570 can include a number of memory slots to store pages ofimage data, as described above, or data corresponding to various modesof operation of image sensor circuit 500, such as operating extendedpixel array 510 in a first mode (e.g., office mode) where a first set ofpixels (e.g., pixels 820) are used during operation, or a second mode(e.g., gaming mode) where a second set of pixels (e.g., pixels 820+830)are used, for instance. Interface 550 can operate to allow image sensorcircuit 500 to communicate with other systems within input device 400.Image sensor circuit 500 may incorporate one or more busses to providepower and/or communicatively connect the various system block of imagesensor circuit 500.

FIG. 6 shows a simplified diagram of aspects of a movement trackingsystem 600 for an input device 400, according to certain embodiments.Movement tracking system 600 can include a light source 610, anillumination lens 620, an imaging lens 660, and an image sensor circuit680. Light source 610 may be any suitable source including an IR LED.Illumination lens 620 can be configured to steer and/or focus a lightbeam from the light source to a target location. In some cases, thetarget location can be a location 650 on an underlying surface while theinput device is operating on the underlying surface, according tocertain embodiments. Imaging lens 660 may be configured to steer and/orfocus the reflected light beam that is reflected off of the underlyingsurface towards a pixel array 670 of image sensor 680 forming a spot690. The image sensor circuit 680 can be configured to generate trackingdata by the image sensor that corresponds to a two-dimensional (2D)movement of the input device with respect to the underlying surfacebased on the received reflected light beam, as further described below.

FIG. 7 shows a pixel array 710 for a conventional image sensor circuit.Pixel array 710 can be a complementary metal-oxide semiconductor (CMOS)sensor that captures images of the underlying surface and sends eachimage to a processor (e.g., processor 210, logic backend 540, etc., toperform image correlation and displacement calculations, etc.) foranalysis. Other types of image sensors may be used, includingcharge-coupled devices (CCD), N-type metal-oxide-semiconductors (NMOS),hybrid devices (e.g., CCD/CMOS), or the like, as would be understood byone of ordinary skill in the art. In conventional designs, pixel array710 it typically comprised of a plurality of individual pixels 705 ofany suitable size that form a square shape. Referring to FIG. 7, a 32×32pixel matrix 420 is shown. Generally, the more pixels used in the pixelmatrix, the more accurate and higher the speed of detected movement canbe, at the cost of greater processing resources (e.g., requiring logicbackend 540 and/or processor 210 to process greater amounts of data) inaddition to greater power consumption. In some cases, some rows andcolumns of pixels may be omitted in tracking calculations to simplifytracking calculations and improve performance. For instance, edge pixelsmay not have similar pixel neighbor conditions as center pixels, whichcan require additional computational resources to reconcile thosedifferences. Thus, in some embodiments, some or all of the data fromedge pixels can be discarded from the correlation computation. One ofordinary skill in the art with the benefit of this disclosure wouldunderstand the many variations, modifications, and alternativeembodiments thereof.

Typically, a light beam is emitted by a light source, reflected off ofan underlying surface, and directed to the pixel array to track amovement of the input device relative to the underlying surface bycapturing “pages” of images of the underlying surface in rapidsuccession and analyzing and comparing a movement of various surfacefeatures (e.g., particles, surface feature variations, etc.) relative tothe input device to determine displacement. Generally, the greater thenumber of pixels used in tracking, the greater the resolution andaccuracy of movement detection at the cost of greater power dissipationand processing bandwidth. Thus, optical devices typically utilize asmany pixels as possible on the image sensor by “overfilling” the pixelarray with the reflected light, which typically forms a roundilluminated “spot” 790 on the pixel array. The alignment of the spot 790on the pixel array is usually done at the manufacturing stage. Imagearrays can be a significant cost in an overall manufacturing cost of anoptically tracked input device, so maximizing the number of usablepixels on an array is also advantageous from a cost perspective as well.Utilizing a spot that is smaller than the pixel array would not beconventionally practiced because it would produce a reduction in imageresolution and tracking accuracy and would be a waste of image sensorcost (for the extra unused pixels) and image sensor “real-estate,” whichcan make up a significant portion of the input device. Thus, if amanufacturer wanted to use a smaller spot, their bill of materials wouldsimply call for a smaller pixel array and corresponding image sensorcircuit. In summary, conventional designs typically overfill the imagesensor to take advantage of as many pixels as possible on the pixelarray for the best possible resolution and tracking experience.

Overfilling the standard square-shaped pixel array with the reflectedlight can be advantageous for tracking 2D movement of the input devicealong an underlying surface, but also can make lift detection andtracking while the input device is lifted using the same pixel arrayunreliable and impracticable. One way to perform lift detection using apixel array is to detect an edge or edges of the spot illuminating thepixel array, determine a centroid of the spot based on the detectededge, and detect how the centroid moves, which may correlate to how muchthe input device has lifted. The edges 792 are detected by identifyingboundaries where a first pixel of a pair of adjacent pixels are at orabove a threshold illumination value (e.g., the spot illuminates thepixel) and a second pixel of the pair of adjacent pixels are below thethreshold illumination value (e.g., the spot does not illuminate thepixel). This can be done at every pair of adjacent pixels that areabove/below the threshold value, as described above, or at a subsetthereof. The centroid can be determined (e.g., interpolated) based onthe identified boundaries. The determination of the edge and centroidcan be performed in a variety of ways, as would be appreciated by one ofordinary skill in the art with the benefit of this disclosure. As shownin FIG. 7, the spot mostly overfills the pixel array (e.g., to achieveoptimal resolution, tracking, and pixel usage efficiency) withboundaries shown near the four corners of the pixel array. When theinput device is lifted even a small amount due to input device tiltingor skating (e.g., 2-5 mm), the spot can move substantially or completelyoff of the square pixel array 710 such that fewer edges or no edges canbe detected, which can lead to spurious tracking, in accurate liftdetection due to the inability to accurately track the centroid, orother deleterious (parasitic) effects. Aspects of the invention relateto an extended pixel array (e.g., pixel array 810) that allow forextended tracking of the centroid resulting in accurate lift detectionover a higher range (e.g., 5 mm or more) and continued 2D tracking alongthe underlying surface while the input device is lifted.

FIG. 8 shows an improve pixel array 810 for an image sensor circuit,according to certain embodiments. Pixel array 810 can be a CMOS, CCD,NMOS, CCD/CMOS, or other suitable type of image sensor technology.Referring to FIG. 8, pixel array 810 is rectangular and includes aplurality of pixels including a first set of pixels 820 (e.g., 32×32pixels) that form a square, and a second set of pixels 830 (e.g., 15×32pixels) configured adjacent to the first set of pixels that extend thesquare in a direction that the centroid moves to as the input device islifted off of the underlying surface. As shown in FIG. 8, the second setof pixels 830 extends the pixel array 810 to the left side of the firstset of pixels 820, however the extension of the pixel array can beconfigured in any direction that preferably allows the pixel array 810to track continue to track the illuminated spot as it moves across thepixel array in correspondence to an amount that the input device islifted off of the underlying surface, as would be appreciated by one ofordinary skill in the art with the benefit of this disclosure.

As described above with respect to FIG. 7, conventional square pixelarrays are overfilled by the illuminated spot (e.g., 1.2:1) such thatthe edges of the illuminated spot can be challenging to locate, as wellas the corresponding centroid, as there typically few edges 792 thatfall within the pixel array, and any movement of the spot due to theinput device being lifted off of an underlying surface would be verydifficult, if not impossible, and would require highly sophisticatedalgorithms and processing power to track the centroid of the illuminatedspot (also referred to as an “illumination spot”) with a requisiteamount of consistency or accuracy because the edges of the illuminatedspot begin to move off of the pixel array as the input device is lifted,and any lift beyond 1-2 mm would make tracking the lift, or much lesstracking 2D movement on the underlying surface, to be practically notpossible as very few pixels, if any, would be illuminated making thetracking resolution and accuracy fall below any acceptable performancelevels.

The extension of the first set of pixels 820 with the second set ofpixels 830 allows the spot to be tracked to 5 mm or more as the edges ofthe illuminated spot can be tracked as the centroid moves from the firstset of pixels to the second set of pixels. In some aspects, the entirepixel array 810 can be tracked at all times and in real-time to ensurethat the illuminated spot can be tracked in response to frequentskating, tilting, etc., while in use. In some aspects, the second set ofpixels may not be read until there is some indication in the first setof pixels that a lift detection condition exists, which gains thebenefit of having a greater working distance (e.g., accurate tracking ina lift detect condition) of the input device, while conserving power bynot accessing/reading the pixels of the second set of pixels 830 untillift detection and tracking is needed. Although two sets of pixels areshown, the additional pixels can be represented in any suitable fashion(e.g., one additional set of pixels, two additional sets of pixels,quadrants of pixels, etc.). One of ordinary skill in the art with thebenefit of this disclosure would appreciate the many modifications,variations, and alternative embodiments thereof.

FIG. 9 shows aspects of lift detection for an input device 400,according to certain embodiments. Input device 400 is shown in a firstposition at time t₁ when input device 400 is operating on an underlyingsurface, and a second position t₂ when input device 400 is lifted and/ortilted (e.g., 2 mm as shown) off of the underlying surface. As describedabove, a light source (e.g., IR LED) projects a light beam that issteered to a target location on the underlying surface. The reflectedlight from the target location is directed to the image sensor, wherethe light preferably overfills and illuminates a first set of pixels820, as shown by spot 990. Spot 990 is shown as a circle, although othershapes are possible (e.g., a square), and preferably a shape thatutilizes the most pixels in the first set of pixels 820 to maximizemovement tracking accuracy and resolution and minimizing the amount ofunused pixels. In conventional devices and in embodiments of the presentinvention, the various lenses (e.g., illumination and image lenses) arealigned at the time of manufacturing to ensure that the spot is centeredon the first set of pixels 820 and utilizes most of the pixels in thefirst set of pixels 820 to ensure good tracking performancecharacteristics (e.g., typically 1.2:1 for light/pixel array ratio).However, aspects of the present invention further include a second setof pixels 830 that is configured to receive the spot as the spot movesin the direction of the second set of pixels as the input device islifted.

Referring back to FIG. 9, at t₁ the location of the spot can be trackedby identifying edges of the spot and then computing a centroid, as wouldbe appreciated by one of ordinary skill in the art with the benefit ofthis disclosure. In some embodiments, the centroid computation andtracking may be performed by system 200, 500, or any suitable system orcombination of systems described herein. As noted above, a movement ofthe centroid of the illuminated spot off of a square image sensor cancause parasitic effects that can manifest as non-user-initiated,spurious cursor movement on a display. Typically, the image sensor isoverfilled to utilize as many pixels as possible for good trackingaccuracy and resolution (e.g., input device movement, velocity, andacceleration). In convention square pixel arrays (e.g., pixel array710), when a user lifts (e.g., skates) or tilts the input device, theilluminated spot moves off of the image sensor. An edge of the spot canbe tracked and typically appears as a transition from a strongillumination signal (the spot) on certain pixels and a sharp decrease inillumination on adjacent pixels. Because fewer pixels are illuminated,tracking accuracy can be reduced (e.g., reduced DPI tracking). Moreconsequentially, in some cases, the pixels that register shade (e.g.,illumination below a threshold value—typically pixels where theillumination spot is not trained upon) may be confused with surfacefeatures, which can create poor tracking conditions and cause parasiticeffects, such as spurious cursor movement. For example, this can be veryapparent when tilting a computer mouse in many contemporary devices.When a user tilts the mouse, there is no real movement of the mouse fromits present position. Ideally, a cursor would not move on the displaywhen the mouse is stationary but tilted, and the tilt would be ignored.In a typical contemporary computer mouse with a square image sensor withillumination overfill, the tilted condition will cause the spot to moveand create pixels with shade. The centroid of the spot may be difficultto ascertain because of the limited number of edge pixels (e.g.,detectable on one side and at most two corners only), which can furthercomplicate tracking and/or any potential correction subroutines that mayproperly evaluate the tilt condition and tracking accordingly (e.g., nocursor movement). For example, if an accurate centroid cannot bedetected, particularly while moving when the input device is lifted, itmay difficult to determine how the input device is moving relative to anunderlying surface because there may be no reliable point ofcorrelation. As an example, features on the underlying surface that areprojected within the illumination spot may appear stationary when theinput device is not moving. When the input device moves, the detectedfeatures may move proportionally, which can inform how to control thecursor in a commensurate manner. However, when the input device islifted, it may appear that an underlying feature is moving when it isactually the spot that is moving, and this could translate into detectedmovement. If the system knows how much the input device is lifted (e.g.,by tracking the centroid), then the system can compensate using thecentroid shift to determine whether movement with respect to theunderlying surface actually occurred, as would be appreciated by one ofordinary skill in the art with the benefit of this disclosure. With theaddition of an extended pixel array, as shown in FIGS. 8 and 9, a mousetilt or lift may cause the illumination spot to move onto the extendedportion of the array. Because the spot can be tracked on both sides(e.g., right and left side of spot), the centroid can more accuratelytracked, and accurate compensation of tracking for the input device canbe performed, which can reduce or eliminate any spurious trackingconditions at least during the range on the entire image sensor wherethe centroid can be accurately tracked.

Referring to FIG. 9, as input device 400 is lifted and/or tilted, thecentroid of the illuminated spot moves from position C₁ at t₁ toposition C₂ at t₂. Note that at each position between C₁ and C₂, thereare four corners of the illuminated spot that are visible, made possibleby the pixels in the extended portion of the pixel array 830. Thus, att₁ and t₂ and every time in between, an accurate centroid of theilluminated spot can be detected, and accurate movement tracking thatcompensates for lift can be performed. The accurate detection throughoutthe LoD range can result in improved DPI in lift-detect conditions,which can result in nominal tracking over a wider range of use. In otherwords, the input device

In some embodiments, input device 400 may operate in different modes ofoperation. For example, in a first mode of operation (e.g., an“productivity” or “office” mode), it may not matter that there is somespurious movement of the cursor for users working in an officeenvironment and using word processing or spreadsheet applications. Insuch modes, the additional (extended) set of pixels 830 may not beactivated, even when a lift detection event is detected. In a secondmode of operation (e.g., gaming mode), accurate tracking may be achievedduring a lift-detect event by employing the second set of pixels 830 atall times to minimize or eliminate spurious tracking during skating,tilting, or lifting actions up to the LoD threshold. In some cases, theLoD can be set by a user, which can be modulated by setting the numberof pixels 830 used to track the illuminated spot. In some cases, LoD maybe set between 1-5 mm, although lower or higher LoD thresholds arepossible, which can be based at least in part on the length of theextended set of pixels 830, as would be appreciated by one of ordinaryskill in the art with the benefit of this disclosure. In someembodiments, switching between modes may be based on a user input (e.g.,pressing a button on the input device that causes the one or moreprocessors 302 and/or logic backend 540 to switch between productivityand gaming modes. In some cases, if the system detects that the inputdevice is operating on a high-contrast surface, such as a mouse pad, theinput device may be switched to gaming mode, and if the system detectsthat the input device is operating on a low-contrast surface (e.g.,clear paper, glass surfaces, desks, or uniform color surfaces), theinput device may be switched to productivity mode. In some cases, thesecond set of pixels may only be used/activated in response todetermining a lift detection condition using the first set of pixels,such that more accurate tracking is possible by incorporating the secondset of pixels during tracking (e.g., resulting in improved DPI during alift/tilt condition), and also have the benefit of power savings by onlypowering/polling the second set of pixels when a lift detect event isinitially and solely detected via the first set of pixels. Anotherbenefit of the extended pixel array is that during manufacturing when athe illumination spot is calibrated and aimed onto the image sensor overarray 820, calibrations that are misaligned can more easily beidentified by using the extended portion of the pixel array 830 to moreaccurately determine the centroid of the illumination spot and determineif the illumination spot needs to be realigned.

FIG. 10 shows a simplified block diagram of a method for performing liftdetection for an input device, according to certain embodiments. Method1000 can be performed by processing logic that may comprise hardware(circuitry, dedicated logic, etc.), software operating on appropriatehardware (such as a general purpose computing system or a dedicatedmachine), firmware (embedded software), or any combination thereof. Incertain embodiments, method 1000 can be performed by aspects of imagesensor circuit 500, processor 210 or aspects of system 200, or acombination thereof.

At operation 1010, method 1000 can include generating a light beam by alight source module controlled by one or more processors of the inputdevice, according to certain embodiments.

At operation 1020, method 1000 can include steering the light beamtowards a target location, wherein the target location corresponds to aspot on an underlying surface while the input device is operating on theunderlying surface, according to certain embodiments. An illuminationlens may be used to steer the light beam towards the target location.

At operation 1030, method 1000 can include steering a reflected lightbeam that is reflected off of the underlying surface towards an imagesensor of the input device, according to certain embodiments. In someaspects, an imaging lens can be sued to steer the reflected beam towardsthe image sensor.

At operation 1040, method 1000 can include receiving the reflected lightbeam by the image sensor, the image sensor controlled by the one or moreprocessors, according to certain embodiments. In some aspects, the imagesensor includes a pixel array having a plurality of pixels that receivesthe reflected light beam on the image sensor. The plurality of pixelscan include a first set of pixels that form a square shape that receivesthe reflected light from the light source module when the input deviceis operating on the underlying surface. The plurality of pixels caninclude a second set of pixels that is adjacent to the first set ofpixels such that the first set of pixels and the second set of pixelstogether form a rectangle. In some aspects, the second set of pixels isconfigured at a location relative to the first set of pixels such thatthe reflected light from the light source module moves from the firstset of pixels to the second set of pixels as the input device is liftedoff of the underlying surface.

At operation 1050, method 1000 can include generating tracking data bythe image sensor that corresponds to a two-dimensional (2D) movement ofthe input device with respect to the underlying surface based on thereceived reflected light beam, according to certain embodiments.

At operation 1060, method 1000 can include determining that the inputdevice is operating on and in contact with the underlying surface whenthe reflected light beam received by the image sensor is located on afirst set of pixels of a plurality of pixels of the image sensor,according to certain embodiments.

At operation 1070, method 1000 can include determining that the inputdevice is operating above and not in contact with the underlying surfacewhen the reflected light beam received by the image sensor is located ona second set of pixels of the plurality of pixels of the image sensor,according to certain embodiments. As described above, the reflectedlight from the light source module can form a spot on the first set ofpixels when the input device is operating on the underlying surface andmethod 1000 can further include detecting an edge of the spot, by theone or more processors, by identifying boundaries where a first pixel ofa pair of adjacent pixels are at or above a threshold illumination valueand a second pixel of a pair of adjacent pixels are below the thresholdillumination value and determining a centroid of the spot, by the one ormore processors, based on the detected edge of the spot. In some cases,method 1000 can further include determining, by the one or moreprocessors, an amount that the input device has lifted off of theunderlying surface based on the location of the determined centroid ofthe spot on the plurality of pixels.

In some embodiments, the input device can further comprise an inertialmeasurement unit (IMU) with an accelerometer, where method 1000 canfurther include determining whether the input device has been liftedvertically from the underlying surface or tilted off of the underlyingsurface based, in part, on inertial data received from the IMU and thelocation of the centroid of the spot on the plurality of pixels.

In certain embodiments, the reflected light substantially overfills thefirst set of pixels and not the second set of pixels while the inputdevice is operating on the underlying surface, and the reflected lightsubstantially overfills the second set of pixels or a portion thereofwhen the input device is lifted or tilted off of the underlying surface.

It should be appreciated that the specific steps illustrated in FIG. 10provide a particular method 1000 for performing lift detection for aninput device, according to certain embodiments. Other sequences of stepsmay also be performed according to alternative embodiments. Furthermore,additional steps may be added or removed depending on the particularapplications. Any combination of changes can be used and one of ordinaryskill in the art with the benefit of this disclosure would understandthe many variations, modifications, and alternative embodiments thereof.

The terms and expressions that have been employed are used as terms ofdescription and not of limitation, and there is no intention in the useof such terms and expressions of excluding any equivalents of thefeatures shown and described or portions thereof. It is recognized,however, that various modifications are possible within the scope of thesystems and methods claimed. Thus, it should be understood that,although the present system and methods have been specifically disclosedby examples and optional features, modification and variation of theconcepts herein disclosed should be recognized by those skilled in theart, and that such modifications and variations are considered to bewithin the scope of the systems and methods as defined by the appendedclaims.

Most embodiments utilize at least one network that would be familiar tothose skilled in the art for supporting communications using any of avariety of commercially available protocols, such as TCP/IP, UDP, OSI,FTP, UPnP, NFS, CIFS, and the like. The network can be, for example, alocal area network, a wide-area network, a virtual private network, theInternet, an intranet, an extranet, a public switched telephone network,an infrared network, a wireless network, and any combination thereof.

In embodiments utilizing a network server as the operation server or thesecurity server, the network server can run any of a variety of serveror mid-tier applications, including HTTP servers, FTP servers, CGIservers, data servers, Java servers, and business application servers.The server(s) also may be capable of executing programs or scripts inresponse to requests from user devices, such as by executing one or moreapplications that may be implemented as one or more scripts or programswritten in any programming language, including but not limited to Java®,C, C# or C++, or any scripting language, such as Perl, Python or TCL, aswell as combinations thereof. The server(s) may also include databaseservers, including without limitation those commercially available fromOracle®, Microsoft®, Sybase® and IBM®.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.), and working memory asdescribed above. The computer-readable storage media reader can beconnected with, or configured to receive, a non-transitorycomputer-readable storage medium, representing remote, local, fixed,and/or removable storage devices as well as storage media fortemporarily and/or more permanently containing, storing, transmitting,and retrieving computer-readable information. The system and variousdevices also typically will include a number of software applications,modules, services or other elements located within at least one workingmemory device, including an operating system and application programs,such as a client application or browser. It should be appreciated thatalternate embodiments may have numerous variations from that describedabove. F or example, customized hardware might also be used and/orparticular elements might be implemented in hardware, software(including portable software, such as applets) or both. Further,connections to other computing devices such as network input/outputdevices may be employed.

Numerous specific details are set forth herein to provide a thoroughunderstanding of the claimed subject matter. However, those skilled inthe art will understand that the claimed subject matter may be practicedwithout these specific details. In other instances, methods,apparatuses, or systems that would be known by one of ordinary skillhave not been described in detail so as not to obscure claimed subjectmatter. The various embodiments illustrated and described are providedmerely as examples to illustrate various features of the claims.However, features shown and described with respect to any givenembodiment are not necessarily limited to the associated embodiment andmay be used or combined with other embodiments that are shown anddescribed. Further, the claims are not intended to be limited by any oneexample embodiment.

While the present subject matter has been described in detail withrespect to specific embodiments thereof, it will be appreciated thatthose skilled in the art, upon attaining an understanding of theforegoing may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, it should be understoodthat the present disclosure has been presented for purposes of examplerather than limitation, and does not preclude inclusion of suchmodifications, variations, and/or additions to the present subjectmatter as would be readily apparent to one of ordinary skill in the art.Indeed, the methods and systems described herein may be embodied in avariety of other forms; furthermore, various omissions, substitutionsand changes in the form of the methods and systems described herein maybe made without departing from the spirit of the present disclosure. Theaccompanying claims and their equivalents are intended to cover suchforms or modifications as would fall within the scope and spirit of thepresent disclosure.

Although the present disclosure provides certain example embodiments andapplications, other embodiments that are apparent to those of ordinaryskill in the art, including embodiments which do not provide all of thefeatures and advantages set forth herein, are also within the scope ofthis disclosure. Accordingly, the scope of the present disclosure isintended to be defined only by reference to the appended claims.

Unless specifically stated otherwise, it is appreciated that throughoutthis specification discussions utilizing terms such as “processing,”“computing,” “calculating,” “determining,” and “identifying” or the likerefer to actions or processes of a computing device, such as one or morecomputers or a similar electronic computing device or devices, thatmanipulate or transform data represented as physical electronic ormagnetic quantities within memories, registers, or other informationstorage devices, transmission devices, or display devices of thecomputing platform.

The system or systems discussed herein are not limited to any particularhardware architecture or configuration. A computing device can includeany suitable arrangement of components that provide a result conditionedon one or more inputs. Suitable computing devices include multi-purposemicroprocessor-based computer systems accessing stored software thatprograms or configures the computing system from a general purposecomputing apparatus to a specialized computing apparatus implementingone or more embodiments of the present subject matter. Any suitableprogramming, scripting, or other type of language or combinations oflanguages may be used to implement the teachings contained herein insoftware to be used in programming or configuring a computing device.

Embodiments of the methods disclosed herein may be performed in theoperation of such computing devices. The order of the blocks presentedin the examples above can be varied—for example, blocks can bere-ordered, combined, and/or broken into sub-blocks. Certain blocks orprocesses can be performed in parallel.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain examples include, while otherexamples do not include, certain features, elements, and/or steps. Thus,such conditional language is not generally intended to imply thatfeatures, elements and/or steps are in any way required for one or moreexamples or that one or more examples necessarily include logic fordeciding, with or without author input or prompting, whether thesefeatures, elements and/or steps are included or are to be performed inany particular example.

The terms “comprising,” “including,” “having,” and the like aresynonymous and are used inclusively, in an open-ended fashion, and donot exclude additional elements, features, acts, operations, and soforth. Also, the term “or” is used in its inclusive sense (and not inits exclusive sense) so that when used, for example, to connect a listof elements, the term “or” means one, some, or all of the elements inthe list. The use of “adapted to” or “configured to” herein is meant asopen and inclusive language that does not foreclose devices adapted toor configured to perform additional tasks or steps. Additionally, theuse of “based on” is meant to be open and inclusive, in that a process,step, calculation, or other action “based on” one or more recitedconditions or values may, in practice, be based on additional conditionsor values beyond those recited. Similarly, the use of “based at least inpart on” is meant to be open and inclusive, in that a process, step,calculation, or other action “based at least in part on” one or morerecited conditions or values may, in practice, be based on additionalconditions or values beyond those recited. Headings, lists, andnumbering included herein are for ease of explanation only and are notmeant to be limiting.

The various features and processes described above may be usedindependently of one another, or may be combined in various ways. Allpossible combinations and sub-combinations are intended to fall withinthe scope of the present disclosure. In addition, certain method orprocess blocks may be omitted in some embodiments. The methods andprocesses described herein are also not limited to any particularsequence, and the blocks or states relating thereto can be performed inother sequences that are appropriate. For example, described blocks orstates may be performed in an order other than that specificallydisclosed, or multiple blocks or states may be combined in a singleblock or state. The example blocks or states may be performed in serial,in parallel, or in some other manner. Blocks or states may be added toor removed from the disclosed examples. Similarly, the example systemsand components described herein may be configured differently thandescribed. For example, elements may be added to, removed from, orrearranged compared to the disclosed examples.

1. An input device comprising: a housing; one or more processors; alight source module coupled to the housing and controlled by the one ormore processors, the light source module configured to generate anddirect light towards an underlying surface that the input device isoperating on; and an image sensor module coupled to the housing andcontrolled by the one or more processors, the image sensor moduleincluding an image sensor configured to: receive reflected light fromthe light source module that is reflected off of the underlying surface;and generate tracking data that corresponds to a two-dimensional (2D)movement of the input device with respect to an underlying surface basedon the received reflected light from the light source module, whereinthe image sensor is comprised of a plurality of pixels including: afirst set of pixels of the plurality of pixels configured to receive thereflected light from the light source module when the input device isoperating on the underlying surface; and a second set of pixels of theplurality of pixels adjacent to the first set of pixels that isconfigured to extend a vertical movement detection range of the inputdevice by receiving the reflected light from the light source modulewhen the input device is lifted off of the underlying surface, theextended vertical movement detection range enabling a generation of 2Dtracking data that facilitates lift detection and 2D movement trackingwhile the input device is lifted off of the underlying surface.
 2. Theinput device of claim 1 wherein the first set of pixels forms a squareshape that receives the reflected light from the light source modulewhen the input device is operating on the underlying surface, whereinthe second set of pixels is adjacent to the first set of pixels suchthat the first set of pixels and the second set of pixels together forma rectangle, and wherein the second set of pixels is configured at alocation relative to the first set of pixels such that the reflectedlight from the light source module moves from the first set of pixels tothe second set of pixels as the input device is lifted off of theunderlying surface.
 3. The input device of claim 1 wherein the inputdevice is configured to detect both 2D movement of the input devicerelative to the underlying surface and detect the input device beinglifted off of the corresponding surface using a single image sensormodule.
 4. The input device of claim 1 wherein reflected light from thelight source module forms a spot on the first set of pixels when theinput device is operating on the underlying surface, and wherein the oneor more processors are configured to: detect an edge of the spot byidentifying boundaries where a first pixel of a pair of adjacent pixelsare at or above a threshold illumination value and a second pixel of apair of adjacent pixels are below the threshold illumination value; anddetermine a centroid of the spot based on the detected edge of the spot.5. The input device of claim 4 wherein the one or more processors arefurther configured to determine an amount that the input device haslifted off of the underlying surface based on a location of thedetermined centroid of the spot on the plurality of pixels.
 6. The inputdevice of claim 5 further comprising an inertial measurement unit (IMU)with an accelerometer, wherein the one or more processors are furtherconfigured to determine whether the input device has been liftedvertically from the underlying surface or tilted off of the underlyingsurface based, in part, on inertial data received from the IMU and thelocation of the centroid of the spot on the plurality of pixels.
 7. Theinput device of claim 1 wherein the light source module includes aninfra-red LED.
 8. The input device of claim 1 further comprising: afirst lens configured to direct light from the light source moduletowards the underlying surface; and a second lens configured to directthe reflected light off of the underlying surface to the first set ofpixels of the image sensor when the input device is operating on theunderlying surface.
 9. The input device of claim 8 wherein the reflectedlight substantially overfills the first set of pixels and not the secondset of pixels while the input device is operating on the underlyingsurface.
 10. The input device of claim 9 wherein the reflected lightsubstantially fills at least a majority portion the second set of pixelswhen the input device is lifted or tilted off of the underlying surface.11. A method of operating an input device, the method comprising:generating a light beam by a light source module controlled by one ormore processors of the input device; steering the light beam towards atarget location, wherein the target location corresponds to a spot on anunderlying surface while the input device is operating on the underlyingsurface; steering a reflected light beam that is reflected off of theunderlying surface towards an image sensor of the input device;receiving the reflected light beam by the image sensor, the image sensorcontrolled by the one or more processors; generating tracking data bythe image sensor that corresponds to a two-dimensional (2D) movement ofthe input device with respect to the underlying surface based on thereceived reflected light beam; determining that the input device isoperating on and in contact with the underlying surface when thereflected light beam received by the image sensor is located on a firstset of pixels of a plurality of pixels of the image sensor; determiningthat the input device is operating above and not in contact with theunderlying surface when the reflected light beam received by the imagesensor is located on a second set of pixels of the plurality of pixelsof the image sensor; and tracking 2D movement of the input device whilethe input device is lifted off of the underlying surface when thereflected light beam received by the image sensor is located on a secondset of pixels of the plurality of pixels of the image sensor.
 12. Themethod of claim 11 wherein the first set of pixels forms a square shapethat receives the reflected light from the light source module when theinput device is operating on the underlying surface, wherein the secondset of pixels is adjacent to the first set of pixels such that the firstset of pixels and the second set of pixels together form a rectangle,and wherein the second set of pixels is configured at a locationrelative to the first set of pixels such that the reflected light fromthe light source module moves from the first set of pixels to the secondset of pixels as the input device is lifted off of the underlyingsurface.
 13. The method of claim 11 wherein reflected light from thelight source module forms a spot on the first set of pixels when theinput device is operating on the underlying surface, and wherein themethod further comprises: detecting an edge of the spot, by the one ormore processors, by identifying boundaries where a first pixel of a pairof adjacent pixels are at or above a threshold illumination value and asecond pixel of a pair of adjacent pixels are below the thresholdillumination value; and determining a centroid of the spot, by the oneor more processors, based on the detected edge of the spot.
 14. Themethod of claim 13 further comprising: determining, by the one or moreprocessors, an amount that the input device has lifted off of theunderlying surface based on the location of the determined centroid ofthe spot on the plurality of pixels.
 15. The method of claim 13 whereinthe input device further comprises an inertial measurement unit (IMU)with an accelerometer, wherein the method further includes: determiningwhether the input device has been lifted vertically from the underlyingsurface or tilted off of the underlying surface based, in part, oninertial data received from the IMU and the location of the centroid ofthe spot on the plurality of pixels.
 16. The method of claim 12 whereinthe input device further includes an illumination lens and an imaginglens, wherein the steering the light beam towards a target location isperformed by the illumination lens, and wherein the steering a reflectedlight beam that is reflected off of the underlying surface towards animage sensor is performed by the imaging lens.
 17. The method of claim12 wherein the reflected light substantially overfills the first set ofpixels and not the second set of pixels while the input device isoperating on the underlying surface, and wherein the reflected lightsubstantially overfills the second set of pixels when the input deviceis lifted or tilted off of the underlying surface.
 18. A system foroperating an input device, the system comprising: one or moreprocessors; one or more machine-readable, non-transitory storage mediumsthat include instructions configured to cause the one or more processorsto perform operations including: generating a light beam by a lightsource module controlled by one or more processors of the input device;steering the light beam towards a target location, wherein the targetlocation corresponds to a spot on an underlying surface while the inputdevice is operating on the underlying surface; steering a reflectedlight beam that is reflected off of the underlying surface towards animage sensor of the input device; receiving the reflected light beam bythe image sensor, the image sensor controlled by the one or moreprocessors; generate tracking data by the image sensor that correspondsto a two-dimensional (2D) movement of the input device with respect tothe underlying surface based on the received reflected light beam;determine that the input device is operating on and in contact with theunderlying surface when the reflected light beam received by the imagesensor is located on a first set of pixels of a plurality of pixels ofthe image sensor; determine that the input device is operating above andnot in contact with the underlying surface when the reflected light beamreceived by the image sensor is located on a second set of pixels of theplurality of pixels of the image sensor; and tracking 2D movement of theinput device while the input device is lifted off of the underlyingsurface when the reflected light beam received by the image sensor islocated on a second set of pixels of the plurality of pixels of theimage sensor.
 19. The system of claim 18 wherein the first set of pixelsforms a square shape that receives the reflected light from the lightsource module when the input device is operating on the underlyingsurface, wherein the second set of pixels is adjacent to the first setof pixels such that the first set of pixels and the second set of pixelstogether form a rectangle, and wherein the second set of pixels isconfigured at a location relative to the first set of pixels such thatthe reflected light from the light source module moves from the firstset of pixels to the second set of pixels as the input device is liftedoff of the underlying surface.
 20. The system of claim 18 wherein theinstructions are further configured to cause the one or more processorsto perform operations including: determining a surface type of theunderlying surface; in response to determining that the surface type isa high contrast surface, utilizing both the first and second set ofpixels for tracking the location of the reflected light beam; and inresponse to determining that the surface type is a low contrast surface,utilizing only the first set of pixels for tracking the location of thereflected light beam.